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ABSTRACT 

The research on response facilitation factors in mail 
surveys is reviewed, and background is developed to enable the 
comparison of results obtained by two analytic approaches, a 
descriptive analysis and a meta~analys is , using the same set of 
studies. Specific criteria for sample and methodology were applied to 
studies extracted from four CD-ROM databases using search terms for 
mail survey combined with response rate. Multiple publications of the 
same study were deleted. The final listing of articles to be used in 
the reviews contained a total of 208 publications. After final 
agreement was reached on the studies that were to be included, the 
two reviews were conducted independently, A list of the 208 sources 
of studies is included, (Contains 11 references,) (SLD) 
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The purpose of this session is to review the research on response facilitation factors in mail 
surveys and to compare the results obtained by two approaches, a descriptive analysis and a meta- 
analysis, using the same set of research studies. 

Reviews on this topic have been conducted in the past using various methods, with some 
discrepancies in the findings (Boser & Clark, 1993). The reasons for the differences in findings may be at 
least partially due to the method used and/ or the specific studies that served as a basis for the reviews. 

The only variables for which there has been universal agreement about their improvement of 
response rates are the use of follow-up contacts and incentives. None of the previous reviews have used 
the same set of research studies as a basis as will be done in this investigation, and it is possible that some 
of the differences in findings may be the result of basing the reviews on different original studies. In 
addition to searching for answers to the long-standing question about which methods are most helpful in 
promoting response rates in mail surveys, this presentation will offer an opportunity to examine the effect 
of the method used to review the findings of multiple studies when the body of studies is held constant. 

Some reviews, like those of Yu and Cooper (1983), have not isolated mail surveys but have 
included studies of response rate improvement techniques in telephone and/or face-to-face interview 
surveys. Still others have given no description of any criteria for the selection of the studies that were 
included in the review (Duncan, 1979; Harvey, 1987; Kanuk & Berenson, 1975), leaving the credibility of 
the findings to the judgment of the reader. Some reviews have focused on a single variable or a set of 
variables decided upon in advance of undertaking the investigation. 

Reviews of research literature on mail survey methods have been facilitated by the development 
of computerized search procedures to locate relevant studies. Recent adaptations including the access of 
researchers to the databases via CD-ROM have further encouraged such reviews. 

The methods used to compare and combine the results of various independent investigations are 
varied and have developed over time. The early reviews, such as those of Linsky (1975) depended on a 
descriptive approach. More recently, regression (Heberlein & Baumgartner, 1978), logit regression 
(Bruvold & Comer, 1988) and meta-analysis (Fox, Crask & Kim, 1988; Yammarino, Skinner, & Childers, 
1991). While more sophisticated analytic approaches may provide more rapid results, they sometimes 
sacrifice specificity by having to collapse categories in order to obtain the numbers necessary for analysis. 
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Method 

For purposes of this investigation, the following criteria were established for studies to be 
included: 

1. The study was focused on identification of variables that facilitate response rates in mail 

surveys. 

2. The study used a split sample approach or an experimentally manipulated variable while the 
other procedures were held constant across groups. 

3. Individuals were randomly or systematically assigned to groups. 

4. The survey was done in the United States with a target population in this country. 

5. The survey reported the response rate for each group. 

6. The sample size was provided. 

7. The written account of the survey was obtainable through published sources and written in the 
English language. 

8. Studies in which the only manipulation was a comparison of mail surveys versus other 
procedures were inadmissible because they did nbt provide ways to improve mail surveys. 

9. Surveys conducted by VAX or disk were not included. 



The reference lists of previously published reviews on this topic were examined for studies that 
fit the inclusion criteria. In particular, reviews by Heberlein and Baumgartner (1978), Fox, Crask and Kim 
(1988), and Yammarino, Skinner and Childers (1991) served as initial sources because they used only 
studies that included an experimental design with one or more manipulated variables. The references 
cited in other reviews, such as those by Linsky (1975) and Bruvold and Comer (1988), were also examined 
for inclusion. 

Computer searches of four CD-ROM databases were conducted using the search terms "mail 
survey" or "Mail surveys" combined with "response rate" or "response rates." The ABI/Inform database 
contains articles in marketing and business publications. Sociofile represents social science, PSYCLit the 
field of psychology, and ERIC the field of education. The number of citations identified in each search are 
listed below: 



Sociofile 


January 1974 through April 1995 


86 


ABI/Inform 


January 1994 - June 1995 


19 




January 1992 - December 1993 


24 




January 1986 - December 1991 


82 


PSYCLit 


January 1990 - June 1995 


37 




January 1974 - December 1989 


88 


ERIC 


January 1992 - March 1995 


48* 




January 1982 - December 1991 


107* 




January 1966 - December 1981 


20* 



^Includes documents on microfilm as well as published articles 



Studies which met the criteria for inclusion were identified by the researchers from the sources 
listed below, and a list of studies was developed. As the researchers coded the studies for analysis, 
additional studies were found not to meet the inclusion criteria. For purposes of analysis it was necessary 
to be able to isolate the effects of a particular variable, either through inclusion of a control group that was 
not subjected to the variable, or a comparison group that received another treatment. 
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Also, multiple publications of the same study were uncovered, and duplicative presentations 
were eliminated so that each study was included only once. The most extreme example is the study that 
had been published in three different journals. The final listing of articles to be used in the reviews 
contained a total of 208 publications. It was not unusual for an article to describe more than one separate 
study, and studies frequently considered more than one independent variable. 

The researchers have made every effort to avoid making inferences and basing their judgments 
only on facts. For instance, two articles in which Faria is the first author do not state the population 
surveyed. The authors are based in Canada, which may well mean that the studies were conducted on 
Canadian populations. Since there was no direct evidence regarding population and nothing in the 
writing that would suggest they were not done in the United States, they were not deleted. 

After final agreement was reached on studies that were to be included, the two reviews were 
conducted independently, one pair of researchers using a meta-analy tic approach, the other a descriptive 
or qualitative approach. 
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